Exploring Alternative Spatial and Temporal Dense Representations for Action Recognition
نویسندگان
چکیده
The automatic analysis of video sequences with individuals performing some actions is currently receiving much attention in the computer vision community. Among the different visual features chosen to tackle the problem of action recognition, local histogram within a region of interest is proven to be very effective. However, we study for the first time whether spatiograms, which are histograms enriched with per-bin spatial information, can be alternatively effective for action characterization. On the other hand, the temporal information of these histograms is usually collapsed by simple averaging of the histograms, which basically ignores the dynamics of the action. In contrast, this paper explores a temporally holistic representation in the form of recurrence matrices which capture pair-wise spatiograms relationships on a frame-by-frame basis. Experimental results show that recurrence matrices are powerful for action classification, whereas spatiograms, in its current usage, are not.
منابع مشابه
Exploring Motion Boundary based Sampling and Spatial-Temporal Context Descriptors for Action Recognition
The most important problem in action recognition is how to represent an action video. The approaches can be roughly divided into four categories: (1) human pose based approaches which utilize human structure information; (2) global action template based approaches which capture appearance and motion information on the whole motion body; (3) local feature based approaches which mainly extract va...
متن کاملEMG-based wrist gesture recognition using a convolutional neural network
Background: Deep learning has revolutionized artificial intelligence and has transformed many fields. It allows processing high-dimensional data (such as signals or images) without the need for feature engineering. The aim of this research is to develop a deep learning-based system to decode motor intent from electromyogram (EMG) signals. Methods: A myoelectric system based on convolutional ne...
متن کاملBeyond Spatial Pyramid Matching: Space-time Extended Descriptor for Action Recognition
We address the problem of generating video features for action recognition. The spatial pyramid and its variants have been very popular feature models due to their success in balancing spatial location encoding and spatial invariance. Although it seems straightforward to extend spatial pyramid to the temporal domain (spatio-temporal pyramid), the large spatio-temporal diversity of unconstrained...
متن کاملMulti-Level ResNets with Stacked SRUs for Action Recognition
Most existing Convolutional Neural Networks(CNNs) used for action recognition are either difficult to optimize or underuse crucial temporal information. Inspired by the fact that the recurrent model consistently makes breakthroughs in the task related to sequence, we propose a novel Multi-Level Recurrent Residual Networks(MRRN) which incorporates three recognition streams. Each stream consists ...
متن کاملTemporal Extension of Scale Pyramid and Spatial Pyramid Matching for Action Recognition
Historically, researchers in the field have spent a great deal of effort to create image representations that have scale invariance and retain spatial location information. This paper proposes to encode equivalent temporal characteristics in video representations for action recognition. To achieve temporal scale invariance, we develop a method called temporal scale pyramid (TSP). To encode temp...
متن کامل